1. Technical Field
This disclosure relates to robotics and prosthetic limbs, including fingertips.
2. Description of Related Art
Robots may be unable to carry on a broad array of important delicate tasks, such as assembly operations, handling fragile objects, and preventing damage in accidental collisions. Prosthetics may similarly be unable to carry on a broad array of important delicate tasks.
Robotic systems may include a rigid set of links with actuated degrees of freedom that are controlled with electric motors. The position, velocity, and/or force at these degrees of freedom (or, in the case of rotating joints, angle, angular velocity, and/or torque) may be controlled by a high-level controller coordinating the movement of each of the joints. The controller may run completely autonomously or be controlled manually in whole or in part by a human operator, as in the case of telerobotics.
Such robotic systems may physically interact with their environment. The high-level controller may obtain knowledge about its environment and respond to both expected and unexpected events. In the case of autonomous systems, machine vision may be employed to identify the location and orientation of objects and precision equipment may direct the robot to a desired location. However, algorithms for machine vision may be slow and prone to errors when vision is obstructed, surfaces are out of view, or there are shadows or is poor contrast.
In telerobotic systems, a human operator may interpret images from a video camera or observe the robot directly and may attempt to adjust his or her commands to the robot as rapidly and precisely as possible. However, this mental concentration may be exhausting to the operator and reaction time may be slow.
Fast-reacting systems may require precise robotics employing stiff mechanical linkages, high-quality position and force encoders, high-speed feedback controllers, and powerful and heavy motors. However, these components can be costly. High impact forces from unexpected collisions with objects in the environment can also be catastrophic to either the robot or the object (including humans who may be in the workspace).
DC motors may drive industrial robotic hands or myoelectric prosthetic hands. The closing speed of an unloaded hand may be directly proportional to the voltage across the motors. In prosthetic applications, this control voltage may be generated from amplified and filtered electromyography recordings of a prosthesis user's residual muscles to produce a voltage in proportion to muscle activation strength to capture a user's intent for opening or closing a hand. See Sears H H, Shaperman J., “Proportional myoelectric hand control: an evaluation,” Am J Phys Med Rehabil. 1991 February, 70(1): 20-8; Kyberd P J, Chappell P H. “The Southampton Hand: an intelligent myoelectric prosthesis,” JRRD, 1994 November, 31(4): 326-34; and Engeberg E D, Meek S., “Improved grasp force sensitivity for prosthetic hands through force-derivative feedback,” Biomedical Engineering, IEEE Transactions on. IEEE, 2008, 55(2): 817-21. However, closing a hand around a rigid object may cause the motors driving the fingertips to stall when they can no longer advance, which can cause high stalling forces (around 50-100 N in prosthetics, and even higher with other robotic hands).
Fragile or compliant objects may therefore be challenging to grasp without damage or deformation because they may not be capable of resisting such forces. If the operator decides to pick up a fragile object, therefore, he or she may need to use a high level of visual attention to precisely time stopping the hand as it encloses on an object before it breaks. Such a task can be quite difficult with prosthetic hands due to delays in visual processing in the cortex (e.g., ˜200 ms), filtering delays in EMG signals, and inertia and friction of motors. This can make timing a precise stopping of fingertips quite challenging.
Stopping too early, on the other hand, may not adequately grip the object (and thus may require additional closing commands). Stopping too late, on the other hand, may crush a fragile object. For prosthetic fingertips that are relatively stiff, there may be little room for forgiveness. As a result, myoelectric prosthetic hand users may avoid grasping these types of fragile objects with their prosthetic hands due to the time-consuming and intense focus that may be required. Surgeons performing telerobotic surgery, as well as other telerobotic operators, may experience similar challenges that may require substantial training and concentration to address.
In other robotic applications, servomotors or stepping motors may be used. But dexterous manipulation of fragile objects may be equally challenging for the same reasons.
A similar control problem may arise in preventing robot appendages from harming themselves or damaging external objects when collisions occur with or between appendages. These appendages may have hard surfaces. The impact force from a collision with a robotic actuator may increase with the hardness of the colliding surfaces and the relative speed of the robotic appendage and the object. By the time a collision is detected by detecting increased loading on actuators, significant damage may have already occurred.
In autonomous robotic applications, the ability to recognize contact may become even more challenging, with robots typically following a prescribed trajectory at full power and stopping only if the object offers substantially high enough resistance to stall the robotic actuators. Machine vision and ultrasonic proximity sensor approaches may help prevent unwanted collisions, but can be subject to high variability in accurately detecting an object before collision. Machine vision in particular can be subject to errors if the cameras become occluded or if lighting is poor. In applications where safety and reliability are of high concern, these technologies may therefore be less desirable.
For prosthetic and robotic fingers, there is a commercially-available product called a BioTac® in which contact is sensed by a pressure sensor connected to a liquid that is used to inflate an elastomeric skin over a rigid core. See U.S. Pat. Nos. 7,658,119, 7,878,075, 8,181,540, 8,401,658, and Fishel J A, Santos V J, Loeb G E. “A robust micro-vibration sensor for biomimetic fingertips,” IEEE/BioRob. IEEE, 2008, pp. 659-63; Wettels N, Santos V J, Johansson R S, Loeb G E. “Biomimetic tactile sensor array,” Adv. Robotics. 2008a, 22(7): 829-49; Wettels N, Smith L M, Santos V J, Loeb G E. “Deformable skin design to enhance response of a biomimetic tactile sensor,” IEEE/BioRob. 2008b, pp. 132-7; Lin C H, Erickson T W, Fishel J A, (null), Wettels N, Loeb G E. “Signal processing and fabrication of a biomimetic tactile sensor array with thermal, force and microvibration modalities,” IEEE/ROB10.2009, pp. 129-34; Wettels N, Loeb G E. “Haptic feature extraction from a biomimetic tactile sensor: force, contact location and curvature,” IEEE/ROBIO. 2011, pp. 2471-8; Fishel J A, Loeb G E. “Bayesian exploration for intelligent identification of textures,” Front. Neurorobot. 2012a, 6; Fishel J A, Loeb G E. “Sensing tactile micro vibrations with the BioTac-Comparison with human sensitivity,” IEEE/BioRob. IEEE, 2012b, pp. 1122-7; Su Z, Fishel J A, Yamamoto T, Loeb G E. “Use of tactile feedback to control exploratory movements to characterize object compliance,” Front. Neurorobot. 2012, 6; and Xu D, Loeb G E, Fishel J A. “Tactile identification of objects using Bayesian exploration,” IEEE International Conference on Robotics and Automation. 2013. The device has a compliance similar to the human fingertip and can provide sensitivity that exceeds human performance. See Fishel J A, Loeb G E. “Sensing tactile micro vibrations with the BioTac-Comparison with human sensitivity,” IEEE/BioRob. IEEE, 2012b, pp. 1122-7. However, the device has a complex electromechanical design and may require an electrically conductive and incompressible liquid, typically a form of saltwater. This may damage mechatronic components if it leaks from the sensor. Further, differences between the inertial properties of the liquid-filled fingertip and the ambient air may amplify vibration from motor actuation so as to create background noise in the pressure measurements. This may make it more challenging to discriminate between mechanical noise and actual contact. Thus, higher thresholds and larger contact forces may be required to produce a contact pressure that exceeds the noise. Additionally, the BioTac and other inflated sensors may require and result in a skin surface that is convex as a result of inflating an elastic skin with a fluid material. However, the shape of an appendage that requires tactile sensing may include complex and compound curves, including regions that are concave, regions that must be subdivided into separately sensed compartments, and regions that project over structures to protect them, but cannot attach to those structures without interfering with their motion. If the fluid inflating the elastic skin escapes, moreover, the skin may lose its desired shape and may collapse onto the underlying rigid structure, thereby losing compliant protection of that rigid structure.
Other tactile sensors have a fluid-filled cavity and/or pressure sensing. Shinoda et al. describe a device that uses air pressure created in carefully shaped channels within a deformable polymer that is conveyed to microphones used as pressure sensors; and differential signals from these multiple sensors are used to extract directional force and slip vibrations. See Shinoda H, Uehara M, Ando S. “A tactile sensor using three-dimensional structure,” ICRA. IEEE, 1993, pp. 435-41. Ringwall and Case describe an array of air-filled channels that convey skin deformation via air pressure to deform a reflective metallic tab for optical detection. See Ringwall C G, Case A W Jr. “Tactile sensor,” Company GE, editor. US Patent Office; 1981. Kim et al. describe an inflatable mouse with a pressure sensor to detect a mouse click. See Kim S, Kim H, Lee B, Nam T J, Lee W. “Inflatable mouse: volume-adjustable mouse with air-pressure-sensitive input and haptic feedback,” CHI 2008. ACM, 2008, pp. 211-24. Dahley et al. describe an air-filled closed-cell elastomeric foam that is electrically conductive for detection of its deformation by contacting electrodes. See Dahley A, Su V, Magnussen B. “Electronic whiteboard system using a tactile foam sensor,” Siemens Technology-to-Business Center LLC, Aktiengesellschaft S, editors, US Patent Office, 2002. Levin and Abramson describe an air-filled bumper with an electrical contact switch to detect collision. See Levin S, Abramson S. “Tactile Sensor,” Friendly Robotics Ltd., editor, WO Patent 2,001,070,541, United States Patent Office, 2002. Ceres et al. describe a pneumatic suction cup for grasping fruit that incorporates a pressure sensor to identify when grasp has been achieved. See Ceres R, Pons J L, Jimenez A R, Martin J M, Calderon L. “Design and implementation of an aided fruit-harvesting robot (Agribot),” Industrial Robot: An International Journal, MCB UP Ltd, 1998, 25(5): 337-46.
Tactile sensors may detect a wide range of physical phenomena, including capacitive, optical, magnetic, inductive, resistive, piezoelectric, piezoresistive, and ultrasonic. See Nicholls H R, Lee M H. “A survey of robot tactile sensing technology,” Intnl J Robotics Res. 1989, 8(3): 3-30; Howe R D. “Tactile sensing and control of robotic manipulation,” Adv. Robotics. 1994, 8(3): 245-61; Lee M H, Nicholls H R. “Tactile sensing for mechatronics—a state of the art survey,”, Mechatronics. 1999, 9:1-31; and Dahiya R S, Metta G, Valle M, Sandini G. “Tactile sensing—from humans to humanoids,” IEEE Trans Robotics. 2010, 26(1): 1-20.
Some tactile sensing approaches have sensor arrays that do not offer much compliance and tend to be insensitive to contact forces applied between discrete tactile cells. Examples of these include the Takktile array (Takktile, LLC), RoboTouch (Pressure Profile Systems), Weiss Tactile Sensors (Weiss Robotics) and the RoboSkin project. Patterning these sensory cells over complex surface can be challenging and costly and may result in areas that are insensitive to contact at the areas where contact sensitivity becomes the most important, such as edges and joints.